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ABSTRACT 

The  U.S.  Joint  Forces  Command  has  been  charged  to  lead  the  transformation  of  the  U.S.  Armed  Forces 
through  development  and  experimentation  of  new  command  and  control  concepts.  In  particular  the 
Knowledge-C2  Working  Group  of  the  Concepts  Division  has  focused  on  four  related  concepts: 

•  Adaptive  Joint  Command  and  Control  (AJC2) 

•  Common  Relevant  Operational  Picture  (CROP),  and  the  Collaborative  Information  Environment 
(CIE) 

•  Joint  Interactive  Planning  (JIP) 

•  Multinational  Operations  (MNOPS) 

This  paper  discusses  these  Knowledge-C2  concepts  and  related  experiments  in  the  U.S.  JFCOM  experimental 
campaign.  It  then  reports  on  the  Unified  Vision  2001  Experiment  (UV01)  results  showing  how  the  HEAT 
metrics  were  used  to  develop  analyses  baselines  and  quantitative  results.  Finally  it  discusses  the  future  of  the 
experimental  campaign  and  events. 

Key  Words:  C2,  Metrics,  Experimentation. 


1.0  INTRODUCTION 

As  the  U.S.  Joint  Forces  Command  pursues  its  mission  of  transforming  the  U.S.  Armed  Forces,  it  has 
embarked  on  an  experimental  campaign  that  is  designed  to  test  new  and  innovative  concepts  for 
transformation.  The  current  integrating,  or  over-aching  concept  is  Rapid  Decisive  Operations  (RDO), 
a  concept  to  achieve  rapid  victory  by  attacking  the  coherence  of  an  enemy’s  ability  to  fight.  It  involves  the 
synchronous  application  of  the  full  range  of  national  capabilities  in  timely  and  direct  Effects-Based 
Operations  (EBO).  It  employs  asymmetric  advantages  in  the  knowledge,  precision,  and  mobility  of  the  Joint 
Force  against  an  adversary’s  critical  functions  to  create  maximum  shock.  Supporting  RDO  and  EBO  are 
several  Knowledge  and  Command  and  Control  (K-C2)  concepts  vital  to  RDO  success.  These  K-C2  concepts 
include: 

•  Adaptive  Joint  Command  and  Control  (AJC2) 

Paper  presented  at  the  RTO  SAS  Symposium  on  “Analysis  of  the  Military  Effectiveness  of  Future  C2  Concepts 
and  Systems ”,  held  at  NC3 A,  The  Hague,  The  Netherlands,  23-25  April  2002,  and  published  in  RTO-MP-1 1 7. 
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ORGANIZATION 


•  Common  Relevant  Operational  Picture  (CROP),  and  the  Collaborative  Information  Environment 
(CIE) 

•  Joint  Interactive  Planning  (JIP) 

•  Multinational  Operations  (MNOPS) 


Figure  1  shows  how  these  and  other  related  concepts  support  RDO  in  the  context  of  a  Small  Scale 
Contingency  (SSC). 
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Figure  1:  A  View  of  the  RDO  Concept  Family. 

This  paper  focuses  on  selected  Knowledge-C2  concepts  and  related  experiments  in  the  U.S.  JFCOM 
experimental  campaign.  It  then  reports  on  the  UV  01  experiment  assessment  results.  In  particular  it  discusses 
the  Headquarters  Effectiveness  Assessment  Tool  (HEAT)  metrics  used  to  develop  analyses  baselines  and 
quantitative  results.  Finally  it  discusses  the  future  of  the  experimental  campaign  and  related  events. 


2.0  THE  KNOWLEDGE-COMMAND  AND  CONTROL  CONCEPTS 

2.1  Adaptive  Joint  Command  and  Control  (AJC2) 

RDO  require  more  responsive  and  coherent  advanced  planning  and  quicker  use  of  capabilities  than  can  be 
accomplished  by  ad  hoc  stand  up  of  a  JTF  headquarters. 

Currently,  when  a  Joint  Task  Force  (JTF)  is  established,  the  crisis  often  has  already  evolved  or  is  close  to 
involving  combat  operations  or  overt  hostile  action  by  the  adversary.  As  crisis  management  passes  from  the 
CINC  to  a  JTF  Commander,  the  coherency  of  plans  and  in-process  actions  may  be  lost.  The  new  JTF 
commander  must  establish  situational  awareness,  create  a  team,  and  establish  processes  that  have  not  been 
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practiced  with  the  new  team.  At  the  most  critical  time  of  a  crisis,  when  small  actions  can  make  large 
differences  in  the  outcome,  the  C2  is  in  danger  of  being  the  most  dysfunctional.  This  is  illustrated  by  the 
bottom  line  in  Figure  2. 


02  Effectiveness 


time) 

Figure  2:  Improving  JTF  Readiness  and  Responsiveness. 

Taking  advantage  of  improvements  in  information  technology,  RDO  AJC2  will  use  networks  touching  every 
part  of  the  force  and  touching  those  that  will  provide  information  and  direction  to  the  force.  These  networks 
will  be  used  before,  during  and  after  the  crisis  for  training,  planning,  and  communication.  Practiced 
collaboration,  habitual  relationships,  and  sharing  of  situational  understanding  will  enable  greater  coherence  of 
C2  and  more  rapid  and  effective  execution.  The  foundation  for  improved  C2  will  be  a  standing  joint  force 
command  and  control  element  in  each  CINC’s  headquarters  (the  top  line  in  Figure  2).  The  C2  element  will 
have  the  equipment,  training  and  authority  to  become  a  backbone  around  which  a  JTF  Commander’s  staff, 
when  stood  up,  will  operate.  Rapidly  deployable,  and  when  augmented,  this  C2  element  will  be  capable  of 
operating  alone  for  a  small  JTF  contingency,  or  operating  as  part  of  a  standing  service  operational 
headquarters  designed  as  a  JTF  or  the  CINC’s  staff,  depending  on  the  size,  scope  and  the  expected  duration  of 
crisis  response  operations.  This  C2  element  develops  an  Operational  Net  Assessment  (ONA),  which  is  an 
in-depth  system  of  systems  analysis  of  a  region  or  potential  adversary  to  identify  and  develop  plans  for  a 
CINC  priority  set  of  selected  missions.  This  process  is  shown  in  Figure  3. 
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Figure  3:  ONA  Process  Elements. 
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ORGANIZATION 


The  coordinated  application  of  national  power  is  enabled  by  a  refined  interagency  collaboration  process  that 
allows  all  partners  to  “inform  and  be  informed  by”  the  others.  The  C2  element  works  closely  with  the 
experimental  interagency  coordination  cell  (now  designated  as  JX)  on  the  CINC’s  staff.  The  value  of  reducing 
or  eliminating  the  ad  hoc  nature  of  IAC  response  and  political/military  coordination  will  be  a  key  element  in 
successful  RDO.  Understanding  that  future  operations  will  be  conducted  in  a  multinational  environment, 
we  will  work  with  our  partners  to  take  advantage  of  the  key  assets,  legitimacy,  and  political  support  they 
provide.  The  challenges  of  policy,  dissimilar  training,  equipment,  technology,  doctrine,  and  language  will  be 
mitigated  by  peacetime  engagement,  training,  and  shared  tools  for  planning. 

In  its  day-to-day  role  the  SJFHQ  will  develop  the  ONA  and  will  have  practiced  those  processes  critical  to 
crisis  management  and  JTF  execution.  They  will  be  prepared  to  respond  with  full  situational  awareness, 
practiced  teamwork,  and  embedded  collaborative  processes  and  tools. 

This  should  provide  several  key  advantages: 

•  Possesses  pre-crisis  knowledge  and  understanding 

•  Becomes  a  high  performance,  well-trained  team  that  understands  the  C2  processes  and  tools 

•  Takes  advantage  of  habitual  relationships  formed  with  CINC’s  staff,  subordinate  commanders, 
and  interagency  and  multinational  participants.  The  SJFHQ  would  maintain  important  “reach-back” 
links  to  U.S.  strategic  planning  and  intelligence  organizations,  non-DoD  agencies 

•  Conduct  distributed  C2  through  multiple  collaborative  networks 
Figure  4  illustrates  one  way  the  organization  could  function. 


SJC2E 

Functions  and 
Internal  Organization 


A  permanent  staff  organization 
Lead  by  a  Flag  Officer 
Mission  Tailorable 
Works  crisis  across:  Influence, 
Deter,  Coerce,  Compel  Defeat 
1  Allows  for  seamless 
planning  and  execution 
1  Applicable  across  the 
Information  1  spectrum  of  conflict 


Five  supporting  teams  working  inside  an  Information  Environment 


Figure  4:  Alternative  Command  Arrangements  for  the  JFHQ. 
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The  AJC2  concept  is  adaptive  in  its  actual  composition  and  in  the  ways  that  the  SJFHQ  could  transition  from 
a  peacetime  posture  to  actual  JTF  operations.  In  the  Millennium  Challenge  2002  major  experiment  (MC  02) 
for  example,  the  CINC  designates  a  Service  component  headquarters  as  the  JTF  headquarters.  This  staff 
organizes  itself  as  a  JTF  command  staff  around  the  knowledge  based  structure  utilized  by  the  SJFHQ. 

2.2  Common  Relevant  Operational  Picture  (CROP) 

The  CROP  is  a  “virtual  information  warehouse”  that  links  to  all  the  available  information  that  the  warfighters 
require.  From  this  virtual  warehouse,  decision  makers  will  tailor  information  displays  that  are  relevant  to  their 
individual  needs.  The  tailored  displays  generated  from  the  common  virtual  warehouse  will  provide  enhanced 
and  shared  Battlespace  awareness  at  all  levels.  This  is  illustrated  in  Figure  5. 


Provide 
information 
that  is 
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GLOBAL 
INFORMATION 
SOURCES 


Figure  5:  The  Common  Relevant  Operational  Picture. 


The  CROP  concept  focuses  on  the  ability  to  access  the  virtual  data/information  warehouse  enabling  a 
presentation  of  timely,  accurate,  and  relevant  information  that  can  be  tailored  to  meet  the  requirements  of  the 
Joint  Force  Commander  and  every  organization  and  individual  operating  in  a  joint  environment.  It  includes 
and  mutually  supports  non-DoD  organizations,  allied/coalition  forces,  and  the  existing  common  tactical 
pictures  of  the  Services.  The  CROP’S  enabler,  the  Global  Information  Grid  (GIG),  interconnects  associated 
processes  and  personnel  for  collecting,  processing,  storing,  disseminating,  and  managing  information  on 
demand  to  warfighters,  policy  makers,  and  support  personnel. 

CROP  is  a  key  enabler  to  Adaptive  Joint  Command  and  Control,  enhanced  battle  space  awareness,  and  Joint 
Interactive  Planning  (JIP).  The  CROP  is  a  functional  concept  that  proposes  the  presentation  of  timely,  fused, 
accurate,  assured,  and  relevant  information  that  can  be  tailored  to  meet  the  requirements  of  the  joint  force 
commander  and  the  joint  force.  The  goal  is  to  find  and  present  only  that  best  set  of  relevant  information  the 
warfighting  commander  needs  to  make  good  decisions  and  to  act.  This  is  critical  because  a  rapid  decisive 
operation  will  be  won  through  rapid,  decisive  actions. 
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ORGANIZATION 


The  CROP  concept  is  sufficiently  robust  and  adaptable  to  accommodate  the  full  range  of  exchange  of 
information  with  non-DoD  organizations  (including  governmental,  international,  and  private)  and  coalition 
forces.  It  embraces  and  mutually  supports  the  existing  common  tactical  picture  of  the  Services.  The  resultant 
presentation  of  information  will  be  rapidly  accessible  by  all  approved  users  and  will  support  the  full  range  of 
military  operations.  Simply  put,  the  CROP  concept  will  attempt  to  define  what  information  needs  to  be 
collected,  how  it  should  be  processed  (analysis  and  fusion),  how  it  will  be  disseminated,  and  how  it  will  be 
presented  in  the  future. 

2.3  Joint  Interactive  Planning  (JIP) 

The  JIP  concept  embodies  the  notion  of  the  use  of  information  superiority  to  rapidly  reach  decision 
superiority.  Previously,  planning  and  execution  have  traditionally  progressed  in  a  distinct  and  sequential 
hierarchy.  The  JIP  concept  addresses  a  parallel  planning  process  within  a  Collaborative  Information 
Environment  (CIE — note  that  the  CROP  and  the  JIP  are  both  parts  of  the  CIE).  This  environment  utilizes 
distributed  collaboration  tools  and  virtual  collaboration  to  facilitate  simultaneity  among  CINC  headquarters, 
Standing  Joint  Force  Headquarters  (SJFHQ),  Joint  Force  staffs,  the  Service  components,  allies,  and  other 
organizations  that  are  separated  by  time,  organizational  boundaries,  and  geography.  The  result  is  a  common 
shared  awareness,  unity  of  effort,  and  better  understanding  of  the  commander’s  intent.  The  JIP  concept 
leverages  the  latest  technology  advances  in  information  and  decision  support  systems  and  processes  such  as 
intelligent  agents,  which  search  secure,  and  open  source  databases  to  find  information  needed  to  support 
planning  and  execution.  Support  systems  will  extract,  fuse,  and  translate  the  data  to  make  it  useful  for 
decisionmakers.  This  is  illustrated  in  Figure  6. 


Legacy  -  Sequential  JIP  -  Parallel  Collaborative 


Figure  6:  The  Joint  Interactive  Planning  Concept. 
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2.4  Multinational  Operations  (MNOPS) 

Multinational  Operations  is  not  a  unique  concept  in  itself — rather  it  is  the  purposeful  inclusion  of 
multinational  and  coalition  considerations  and  requirements  in  all  the  other  supporting  concepts.  To  facilitate 
MNOPS  we  have  commenced  a  series  of  multinational  Limited  Objective  Experiments  (LOE)  as  depicted  in 
Figure  7.  Note  that  MNOPS  will  be  included  in  the  major  experiment,  Olympic  Challenge  2004  (OC  04). 


Multinational 
LOE  I 


Olympic 
Impact  02 


Multinational 
LOE  II 


Olympic 
Vision  03 


Multinational 
LOE  III 


LOEs  are  key  to  Multinational  Participation  in 
Olympic  Challenge  2004 


Figure  7:  The  Multinational  LOE  Relation  to  Olympic  Challenge  2004. 

The  first  MNOPS  LOE  focused  on  JIP  by  comparing  an  interactive  planning  process  embodied  in  the  JIP 
process  with  a  more  traditional  sequential  planning  process.  Subsequent  LOE  will  focus  on  information 
sharing  and  collaboration. 


3.0  THE  U.S.  JOINT  FORCES  COMMAND  EXPERIMENTAL  CAMPAIGN 

Figure  8  depicts  the  J9  Experimental  Campaign  Plan.  In  the  context  of  RDO  it  focuses  along  two  distinct  but 
related  pathways  -  Millennium  Pathway  and  Olympic  Pathway.  The  Millennium  Pathway  concentrates  on 
near-term  transformation  realizing  the  requirements  to  use  weapons  systems  that  are  already  in  our  arsenal  or 
in  production.  The  focus  therefore  is  on  how  to  conduct  a  Rapid  Decisive  Operation  in  this  decade. 
The  Olympic  Pathway,  in  contrast,  is  an  integrated  set  of  experiments  that  will  examine  how  the  joint  force 
can  conduct  RDO  in  the  next  decade.  While  the  Millennium  Pathway  is  centered  on  using  today’s  major 
platforms  more  effectively  through  a  greater  degree  of  operational  jointness,  the  Olympic  Pathway  will 
consider  what  capabilities  we  should  develop  and  acquire  as  we  replace  today’s  systems  over  the  next  decade. 
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ORGANIZATION 


Figure  8:  The  J9  Experimental  Campaign  Plan. 


Both  pathways  will  utilize  series  of  wargames  and  seminars  to  frame  issues,  simulations  to  integrate  concepts 
and  capabilities,  and  live,  limited  objective  and  major  experiments.  However  since  many  of  the  Olympic 
Pathway  capabilities  to  be  assessed  do  not  yet  exist,  the  Olympic  Pathway  will  rely  more  heavily  on  the  use  of 
simulations,  surrogates,  virtual  portrayals,  and  prototypes.  Lessons  learned  and  finding  developed  from  the 
Millennium  Pathway  will  be  incorporated  into  the  Olympic  event  planning  and  execution. 


4.0  EXPERIMENTAL  ASSESSMENT  METHODS 

Experimental  events  prior  to  Unified  Venture  01  (UV01)  were  assessed  by  subjective  observation,  participant 
surveys,  and  the  traditional  Army  After  Action  Review  (AAR)  processes.  UV01  was  the  first  major 
experiment  to  be  analyzed  in  detail  and  to  assess  objective  data.  The  subjective  data  consisted  of  observations 
from  retired  General/Flag  Officer  Senior  Concept  Developers,  senior  members  of  the  Interagency  Community 
(to  include  two  former  U.S.  Ambassadors),  participants,  and  data  collectors,  as  well  as  participant  survey 
responses.  Members  of  the  JSJFCOM  Joint  Futures  Lab  analyzed  the  data  and  developed  the  experiment 
findings.  In  addition  quantitative  data  was  collected  and  analyzed  through  the  HEAT  methodology  as 
described  below.  The  complete  analysis  is  contained  in  the  UV01  Final  Report  available  on  the  USJFCOM 
Web  site. 

4.1  The  Headquarters  Effectiveness  Assessment  Tool  (HEAT) 

The  Headquarters  Effectiveness  Assessment  Tool  (HEAT)  is  a  joint  Command  and  Control  (C2)  assessment 
tool  that  has  been  applied  to  over  250  different  headquarters  involved  in  dozens  of  exercises,  experiments, 
and  real  world  operations.  By  defining  the  C2  system  as  an  adaptive  control  system  and  separating  C2 
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measures  of  effectiveness  (MOE)  from  the  supporting  measures  of  performance  (MOP),  HEAT  has  proven  to 
provide  a  robust  capacity  to  both  understand  the  quality  of  C2  processes  and  to  diagnose  the  sources  of  C2 
problems  so  they  can  be  addressed  effectively.  Figure  9  illustrates  the  HEAT  Cycle. 


Figure  9:  The  HEAT  Analytic  Structure. 


A  headquarters  operates  in  a  complex  environment  that  includes  among  other  things,  own  and  adversary 
forces,  the  physical  environment,  and  all  other  relevant  factors  and  players.  By  monitoring  the  environment, 
headquarters  staffs  achieve  situational  understanding  from  which  they  can  develop  courses  of  action  (COA) 
and  predict  the  consequences  of  these  COA.  The  Commander  then  chooses  a  course  of  action  and  issues  a 
directive,  which  in  turn  causes  an  effect  on  the  environment  that  the  headquarters  monitors.  While  this 
description  implies  a  cyclical  operation,  the  impact  of  information  technologies  and  collaboration  tools 
enables  many  of  the  steps  to  be  developed  in  parallel  rather  than  sequentially. 

HEAT  metrics  can  be  taken  at  any  point  in  the  cycle  and,  also  measure  the  degree  and  effectiveness  of 
collaboration.  For  example,  in  an  experiment  or  exercise  where  there  is  an  established  ground  truth  known 
about  the  environment,  HEAT  can  measure  the  quality  of  the  monitoring  process  as  a  percent  of  ground  truth 
(e.g.,  just  how  accurate  is  the  headquarters’  displays  of  own  and  enemy  forces).  In  a  similar  vein,  HEAT  can 
measure  the  accuracy  and  quality  of  the  understandings  that  the  staff  and  commander  derive  from  the 
monitoring  (e.g.,  “the  enemy  is  preparing  to  attack”). 

Because  of  the  structure  of  the  UV  01  experiment,  HEAT  metrics  were  not  applicable  at  every  point  in  the 
cycle.  This  was  primarily  due  to  daily  scenario  time  jumps,  and  the  fact  that  much  of  the  experiment  was 
heavily  scripted  to  support  experiment  objectives  (e.g.,  development  of  the  Operation  Net  Assessment  (ONA) 
and  Effects  Tasking  Order  (ETO).  Accordingly  the  HEAT  team  focused  analyses  on  understanding  quality, 
decision  cycle  times,  and  collaboration  effectiveness.  The  hypothesis  was  that:  “A  richly  connected  JTF  Staff 
will  develop  and  maintain  high  quality  situational  awareness  (SA).”  The  metrics  to  support  this  hypothesis 
would  be  strong  scores  for  understanding  quality  and  frequent  statements  of  “understandings.” 
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4.2  Results:  Understandings 

An  “understanding”  is  a  situation  assessment  which  can  be  thought  of  as  a  hypothesis  or  set  of  hypotheses 
dealing  with  past,  current  and  future  situations.  These  are  scored  as  Correct  (actual  situation  matches  primary 
hypothesis),  Not  Incorrect  (actual  situation  is  included  in  contingencies),  or  Incorrect  (actual  situation  is  not 
considered).  Figure  10  summarizes  the  results  of  91  “understandings”  observed,  recorded,  and  scored  in  the 
experiment. 


Time  Period 

Weighted  %  Correct 

n 

Week  1 

93% 

21 

Week  2 

86% 

42 

Week  3 

69% 

28 

Total 

83% 

91 

-  Weight  n  =  #  Understandings  scored 

•  Correct  =  1 

•  Not  Incorrect  =  .8 

•  Incorrect  =  0 

•  Decline  of  scores  over  time  normal  as  OPTEMPO  increases 

•  High  scores  during  week  1  indicates  employment  of  core  JTF  as  part  of 
CINC  staff  effective  in  that  evidence  of  “learning  curve”  was  not  present 

•  Pre-scripted  Red  during  week  3  reduced  #  of  cases  that  could  be  scored 

Figure  10:  Understanding  Quality. 

The  high  percentage  for  weighted  correct  scores  in  Weeks  1  and  2  indicate  that  the  standing  JFHQ  resident 
within  the  CINC  staff  promotes  early  effectiveness  and  significantly  reduces  the  “learning  curve”  usually 
associated  with  a  newly  formed  JTF.  The  decline  noted  during  week  3  can  be  attributed  to  the  increased 
OPTEMPO  (3  separate  execution  vignettes)  plus  the  fact  that  the  execution  events  were  all  pre-scripted  and 
played  out  in  simulation.  The  primary  focus  of  the  staff  during  this  period  was  to  update  the  ETO.  There  was 
little  focus  placed  on  the  actual,  scripted  execution. 

The  “understandings”  results  tabulated  above,  while  useful  in  themselves  are  much  more  relevant  when 
compared  to  a  baseline.  There  were  no  direct  baselines  for  this  particular  new  innovative  concept.  However, 
since  these  comparisons  are  important  and  provide  great  value  for  data  interpretation,  the  HEAT  team  utilized 
the  deep  store  of  previous  HEAT  data  and  derived  two  “Surrogate  Baselines.”  These  are  presented  in  Figure  1 1 . 
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Quality  of  Situational  Awareness 


Surrogate  Baselines 

Day  1 

2 

3 

4 

5 

Overall 

Joint  Command 
Exercise  in  the  1980s 

.52 

.62 

.52 

.60 

.65 

.60 

Median  % 
Correct 

US  Army  Divisions 
circa  1990  2 

.74 

.82 

.84 

.82 

.82 

.81 

Average  % 
Correct 

UV01 

Week  1 

Week  2 

Week  3 

Overall 

Weighted 

.93 

.86 

.69 

.83 

%  Correct 

Interpretation: 

•  UV01  appears  to  have  achieved  higher  understanding  quality  than  surrogate  baselines 

•  Unlike  the  Surrogates,  understanding  quality  in  UV01  declines  over  time 

•  Time  pressure  is  greater  in  later  stages  of  UV01 

•  Goal  for  future  experimentation  should  be  stable  high  %  correct 

•  UV01  understandings  cover  full  range  of  PMESI;  Surrogates  focused  on  military  issues 


1  -  “HEAT  and  the  Warrior  Preparation  Center”,  Defense  Systems  Inc.,  August  1985 

2  -  “Use  of  ACCES  and  JESS  data  to  Support  Analyses  of  the  Relationship  of  Command  Staff  Performance  and  Battle  Outcomes”,  Defense 
Systems  Inc.,  July  1988. 


Figure  11:  Surrogate  Understandings  Baseline. 


Two  surrogate  baselines  were  examined.  The  first  was  developed  from  Joint  Command  and  Control  Exercises 
conducted  by  various  staffs  in  the  1980s.  These  were  conducted  over  5-day  periods  and  show  strong  evidence 
of  the  “learning  curve”  effect.  The  second  surrogate  was  derived  from  10  Army  Division  level  field  exercises 
and  shows  a  less  pronounced  learning  curve  and  a  significantly  higher  level  of  understanding  quality. 
When  experiment  data  are  compared  to  these  surrogates,  an  even  higher  level  of  understanding  quality  is 
noted,  particularly  during  Weeks  1  and  2.  The  relevance  of  the  Week  3  scores  is  viewed  as  qualitatively 
different  than  those  for  Weeks  1  and  2,  because  of  the  shift  from  effects  based  planning  to  the  three  separate 
execution  events.  In  any  case,  the  data  support  higher  understanding  quality  in  the  experiment.  It  is  also 
important  to  note  that  the  surrogate  baselines  focused  only  on  military  issues  while  experiment  understandings 
covered  the  full  range  of  PMESI  (political,  military,  economic,  social,  infrastructure)  as  shown  in  Figure  12. 
The  data  also  point  out  that  a  goal  for  future  experiments  should  be  a  stable  high  percentage  of  correct 
understandings  to  support  Effects  Based  Operations  (EBO). 
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PMESI  Count  -  Percentage  of  Understandings  for  each  category  /  %  Correct 


Time 

Period 

P 

M 

E 

S 

I 

n 

Week  1 

41%/. 83 

49%  /  .98 

8%/ 1.0 

2%  /  - 

0%  /  - 

49 

Week  2 

32%  /  .88 

53%  /  .84 

7%  /  .80 

3%  /  - 

5%  /  - 

81 

Week  3 

21%/. 33 

71%/. 74 

4%  /  - 

4%  /  - 

0%  /  - 

48 

Total 

32%  /  .79 

55%  /  .84 

7%  /  .90 

3%  /  - 

2%  /  - 

178 

n  =  total  number  of  Understandings  recorded 
=  none  scored 

•  Large  increase  in  percentage  of  military  during  execution  (Week  3)  phase 

•  Few  understandings  expressed  regarding  economic,  social  and 
infrastructure 

•  Increasing  military  percentage  over  time  trades  off  with  percentage  correct 
over  time 

•  Focus  on  military  understandings  suggests  the  presence  of  “tunnel  vision” 
which  would  threaten  PMESI  analyses  and  Effects  Based  Operations 


Figure  12:  Understandings  Across  the  PMESI  Effects. 

UV-01  explored  new  ground  by  its  integrated  focus  on  the  effects  of  the  total  range  of  U.S.  national  power 
across  the  continuum  of  peacetime  environmental  shaping  efforts,  to  disaster  relief,  to  a  small-scale  military 
contingency.  A  large  number  (178)  of  understandings  were  achieved  across  the  PMESI  domains.  Many  of 
these  understandings  could  not  be  scored  (e.g.,  correct,  not-incorrect,  incorrect)  because  of  the  scripted  nature 
of  the  experiment  and  the  scenario  time  jumps.  The  data  also  show  that  the  participants’  focus  was  primarily 
on  the  military  and  political  domains.  As  the  scenario  developed  the  “tunnel  vision”  phenomenon  was 
observed  as  the  participants  focused  more  and  more  on  military  action,  particularly  in  Week  3.  This  explains 
in  good  measure  the  low  number  of  understandings  in  the  economic,  social  and  infrastructure  domains. 
Nevertheless,  successful  EBO  require  a  continuing  focus  on  PMESI  at  the  operational  and  strategic  levels. 
This  indicates  a  need  to  develop  tools  that  will  help  the  participants  maintain  focus  across  the  PMESI 
spectrum  and  avoid  tunnel  vision. 
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Percentage  of  Understandings  that  considered  past, 
present  and  future  situations 


Time 

Period 

Past 

Present 

Future 

Week  1 

4% 

54% 

42% 

Week  2 

3% 

48% 

49% 

Week  3 

5% 

53% 

42% 

•  Less  than  half  of  Understandings  considered  the  future 

•  Not  ideal  to  support  planning  or  Effects  Based  Operations 

Time  Horizon  -  amount  of  time  considered  in  future  Understandings 

-  Example:  “After  today,  Red  capability  to  deploy  TBMs  will  be  reduced”  (24  hours) 

-  Mean  future  time  horizon:  2.3  Days  (game  time) 

-  Median  future  time  horizon:  2  Days  (game  time) 

Conclusion:  These  time  horizons  are  more  appropriate  for  the  tactical  than 
the  operational  level 


Figure  13:  Understanding  Time  Horizons. 


4.3  Time  Horizons/Decision  Cycle  Times 

Understanding  time  horizons  are  important  because  they  indicate  how  far  into  the  future  the  staff  is  projecting 
the  implications  of  its  planning  and  directives,  as  well  as  the  possible  impact  of  what  is  already  past.  In  the 
experiment  less  than  half  of  the  understandings  considered  the  impact  of  that  understanding  on  future 
operations.  The  median  future  time  horizon  of  only  2  days  recorded  in  the  experiment  is  more  appropriate  to 
the  tactical  level  and  much  less  so  to  the  operational  and  strategic  levels,  and  definitely  not  ideal  to  support 
EBO  planning. 
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Average  Cycle  Time  (hours) 

Week  1 

29 

Week  2 

27.5 

Week  3 

21 

Overall 

25.1 

Median  Cycle  Time  (hours)  n 
21.5/24.5  6 

23.5  3 

23  7 

23/23.5  16 


•  Cycle  times  above  represent  real  time 

•  Cycle  times  based  on  game  time  have  little  meaning  due  to  time  jumps 
between  game  days 

•  Doesn’t  account  for  events  occurring  during  jumps 

•  Doesn’t  factor  in  overnight  dead  time 


Conclusion:  The  Decision  Cycle  Times  were  stable  across  the 
experiment  rather  than  shorter  during  the  execution  phase 

Figure  14:  Decision  Cycle  Times. 


Decision  cycle  times  are  important  indicators  of  staff  efficiency,  connectivity,  and  collaboration.  The  average 
and  median  times  listed  in  Figure  14  tend  to  be  artificially  long  because  of  the  scripted  game  time  jumps, 
and  the  overnight  dead  time.  Nevertheless,  the  experiment  cycle  trends  provide  useful  insights.  While  the 
average  cycle  times  showed  an  expected  compression,  the  median  times  did  not,  indicating  that  the  cycle 
times  were  essentially  stable  across  the  experiment  rather  than  shorter  during  execution  events.  Some  cycle 
times  were  particularly  long.  Examples  of  long  cycle  times  are  provided  in  Figure  15. 


•  Decision 

-  Required  TMD  defense  of  Green.  First  indication  at 
261545Feb,  second  when  Green  demands  Patriots 
(171309Mar),  TMD  COA  developed  01 1 120Apr. 
Total  34  days  (Game),  68  1/2  hours  (Real) 

•  Understanding 

-  At  010900APR  (14  May-Real)  CJTF  states  need  to 
understand  relationship  between  Red  government  and 
JTF-S.  At  030900May  (21  May-Real)  relationship 
still  not  clear.  Total  32  days  (Game),  7  days  (Real) 

•  Cycle  times  possibly  could  have  been  shortened  had  State 
been  represented  during  initial  planning  sessions  when 
these  items  were  discussed 

Figure  15:  Long  Decision  Cycle  Times. 
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These  long  decision  cycle  times  are  symptomatic  of  the  need  to  enhance  the  participation  of  State  and  other 
relevant  agencies  in  the  experiment.  The  interagency  process  in  effects  based  operations  at  the  operational 
level  is  an  evolving  process  and  the  experiment  clearly  demonstrated  that  there  is  much  to  learn  if  we  are  to 
fully  exploit  EBO  in  the  context  of  Rapid  Decisive  Operations  (RDO). 

4.4  Collaboration 

A  rich  collaboration  environment  is  essential  to  support  the  RDO  and  EBO  concepts.  During  the  experiment 
there  were  188  collaboration  sessions  documented.  However,  not  every  data  element  of  each  session  was 
captured  on  the  data  collection  sheets.  This  resulted  in  differing  frequencies  of  responses/results  for  methods, 
tools  used,  and  purposes  of  collaboration  session.  For  example,  data  regarding  collaboration  purpose  was 
recorded  for  116  out  of  a  total  of  188  collaboration  sessions  because  of  incomplete  information  on  the  data 
collection  forms.  This  was  not  because  of  data  collection  errors  or  omissions,  but  rather  because  the  specific 
information  (collaboration  purpose  in  this  example)  was  either  not  explicit  or  obvious,  and  the  data  collectors 
were  trained  to  be  unobtrusive  to  not  interfere  with  the  experiment. 

The  collaboration  summaries  shown  in  Figures  16  and  17  show  interesting  trends.  The  large  amount  of 
face-to-face  collaboration  in  Weeks  1  and  2  as  illustrated  in  Figure  16  is  attributed  to  two  key  facts. 
First,  the  DCTS  (Defense  Collaborative  Tool  Set)  was  fragile  and  its  capabilities  built  slowly  over  the  first 
two  weeks  of  the  experiment.  Second,  as  the  participants’  became  more  comfortable  and  familiar  with  the 
on-line  tools,  they  found  ways  to  use  them  more  effectively  (learning  curve  and  cultural  bias  shift). 


Method 

Week  1 

Week  2 

Week  3 

Face  to  Face 

23  (50%)* 

24  (57.1%)* 

18  (32.7%) 

On-Line 

23  (50%)* 

18(42.8%) 

37  (67.2%)* 

Tools** 

Whiteboard 

3  (10.3%) 

5  (25.0%) 

2  (4.8%) 

File  Sharing 

13  (44.8%)* 

13  (65.0%)* 

3  (7.3%) 

Chat 

1  (3.0%) 

0 

8  (19.5%) 

IVOX 

Not  Avail. 

Not  Avail. 

12  (19.7%)* 

E-Mail 

0 

0 

5  (12.2%) 

VTC 

1  (3.0%) 

2  (10.0%) 

1  (2.4%) 

Other 

11  (37.9%) 

0 

10  (24.4%) 

Not  Recorded 

22 

24 

20 

-  Frequency  of  face-to-face  was  consistent  over  first  2  weeks  then 
declined  in  third  week  as  online  interactions  increased 

*  Indicates  most  frequently  occurring  activities  &  tools  used 
**  Reported  collaboration  sessions  only 

Figure  16:  Collaboration  Summary  of  Events  (1  of  2). 
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Week  1 

Week  2 

Week  3 

Total 

Collaboration  Purpose 

Decision  Making 

1  (2.6%) 

0 

3  (6.1%) 

4 

Situation  Awareness 

5  (13.2%) 

9(31.0%) 

16  (32.6%)* 

30 

Planning 

28  (73.7%)* 

19  (65.5%)* 

22  (44.9%)* 

69 

Other 

4  (10.5%) 

1  (3.4%) 

8  (16.3%) 

13 

Problems  Reported 

Procedure 

0 

0 

3(23.1%) 

3 

Training 

1  (16.7%) 

0 

0 

1 

System 

4  (66.7%) 

4  (80.01%) 

10  (76.9%) 

18 

Personal 

0 

1  (20.0%) 

0 

1 

Other 

1  (16.7%) 

0 

0 

1 

-  Planning  Collaboration  was  most  frequent,  with  SA  2nd  most  common 

-  Planning  Collaboration  Activities  decreased  over  time,  as  Situation 
Awareness  Collaboration  Activities  increased  over  time 

-  Observers  did  not  focus  on  or  report  many  problems;  most  were  system 
problems 

*  indicates  most  frequently  occurring  activities 


Figure  17:  Collaboration  Summary  of  Events  (2  of  2). 

Collaboration  was  most  utilized  during  planning  and  for  situational  awareness.  As  the  experiment 
focus  shifted  to  execution  during  Week  3,  situational  awareness  collaboration  increased  significantly. 
Problems  reported  with  collaboration  were  mostly  associated  with  the  DCTS  system,  however  the  learning 
curve  evidence  also  indicates  that  more,  up-front  training  would  have  been  valuable.  Figure  18  shows  some 
examples  of  how  collaboration  made  significant  differences  for  both  planning  and  execution,  and  offers  the 
insight  that  the  collaborative  environment  is  essential  for  rapid  planning  and  execution. 

Week  1  -  COA  #2  chosen  by  CINC  within  23  minutes  of  COA 

alternatives  being  developed.  Short  cycle  time  possible  because 
CINC  represented  in  COA  development  process. 

Week  3  -  CINC  disapproval  of  use  of  C-l  17s  in  assault  operations.  This 

became  known  to  planners  during  ETO  1C  development,  enabling 
work-around  using  “chat”  feature  to  be  constructed  during  planning 
process — CINC  was  represented  during  planning  collaboration 
process. 

Week  3  -  CINC  representative  notices  that  CPCM  sites  not  incorporated  in 
ETO  IB  during  planning. 

Week  3  -  Dynamic  retasking  of  ISR  assets  to  identify  hostile  intent  of 
swarming  boats  accomplished  in  15  minutes  as  a  result  of 
collaboration  between  SOF  OPS,  CINC  JIC,  JTG  OPS  Director, 

JSOTF. 


Figure  18:  Collaboration  Helped  Examples. 
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4.5  Summary 

The  hypothesis  that  “A  richly  connected  JTF  Staff  will  develop  and  maintain  high  quality  situation 
awareness,”  was  supported.  When  compared  to  surrogate  baselines  of  both  command  post  (CPX)  and  field 
(FTX)  exercises,  the  experiment  understanding  quality  was  clearly  better.  This  enhanced  quality  is  particularly 
relevant  given  the  experiment  scripting  and  time  jumps,  which  would  tend  to  reduce  understanding  quality 
because  of  the  lack  of  situational  continuity.  The  absence  of  the  learning  curve  effect  is  a  strong  indication 
that  the  core  STFHQ  concept  is  on  the  right  track. 

The  collaborative  environment  and  resulting  collaboration  are  the  links  that  provided  the  “rich  staff 
connection.”  As  the  collaboration  tools  became  more  robust  over  the  course  of  the  experiment,  and  the  JTF 
staff  became  more  adept  with  the  tools  and  more  comfortable  working  in  the  collaboration  environment, 
planning  errors  and  decision  cycle  times  were  reduced.  Collaborative  planning  started  immediately  while 
situational  awareness  collaboration  developed  over  time.  Further  training  and  experience  with  the 
collaboration  tools  should  further  improve  collaboration. 

While  the  empirical  data  indicate  that  the  time  horizon  was  short  of  ideal,  this  may  well  be  an  artifact  of  the 
experimental  design.  Extensive,  heavy  scripting  and  time  jumps  caused  the  participants  to  be  faced  with  a  new 
situation  every  day.  Under  these  conditions,  it  is  difficult  to  achieve  and  maintain  coherent  time  horizon 
continuity. 

Likewise,  the  long  decision  cycle  times  observed  were  attributed  to  both  the  experiment  design  and  the  need 
for  greater  and  better-integrated  interagency  involvement  in  the  experiment.  Effects  Based  Operations  cannot 
achieve  their  potential  through  military  actions  alone.  There  must  be  strong  linkage  and  involvement  by  the 
other  relevant  actors  to  maintain  the  effects  focus  and  to  avoid  tunnel  vision.  Future  experiments  should 
emphasize  interagency  involvement  in  the  EBO  process. 


5.0  THE  ROAD  AHEAD 

While  the  terrorist  attacks  of  September  11,  2001  have  caused  increased  emphasis  on  homeland  security, 
the  U.S.  Joint  Forces  Command  is  still  fully  committed  to  the  transformation  of  the  U.S.  Armed  Forces 
through  Joint  Experimentation.  The  Experimental  Campaign  Plan  may  receive  some  fine-tuning,  however  the 
basic  goals  and  milestones  will  go  forward  as  planned.  Quantitative  analysis  utilizing  HEAT  will  remain  an 
integral  part  of  the  assessment  methodology. 
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ORGANIZATION 


This  page  has  been  deliberately  left  blank 


Page  intentionnellement  blanche 
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Concepts 


OIUjAMZAIIOK 


A  View  of  the  RDO  Concept  Family 


Full  Range  of  Military  Operations 


MSCA 

HA  /  DR  Peace  Homeland 

ssc 

MTW 

(Military  Support 

(Humanitarian  Assistance  /  Ops  Defense 

(Small  Scale 

(Major  Theater 

To  Civil  Authorities) 

Disaster  Relief) 

Conflict) 

Warfare) 

RDO 

Operational  Net  Assessment 

Common  Relevant  Operational  Picture 

Joint  Intelligence.  Surveillance.  &  Reconnaissance 

Adaptive  Joint  Command  and  Control 

Joint  Interactive  Planning 

Interagency  Operations 

Multinational  Ops 

Effects-based  Operations 

Dominant  Maneuver 

Precision  Engagement 

Information  Operations 

Assured  Access 

Rapid  Force  Deployment 

Agile  Sustainment  Operations  (Focused  Logistics! 

Full  Dimensional  Protection 


RDO 

Effort 

To-date 


SPACE 


OILMAN  AZAT  I  OK 


Improving  JTF  Readiness  and 

Responsiven  ess 


C2  Effectiveness 


Time  into  Mission  (employment  &  execution 
time) 
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ONA  Process  Elements 


ONA  PROCESS 


Pull  &  Organize 

Available 

Information 

•  Political 

•  Military 

•  Economic 

•  Social 

•  Infrastructure 


ID  Potential 
Source  &  Area 
of  Conflict 


Network  Nodal 
Analysis 


ID  Key  Nodes 
&  Links  in 
System  of 
Systems 


Potential  Effects 
Through 
Application  of 

•  Diplomatic 

•  Information 

•  Military 

•  Economic  Power 
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O  KtiAMZA’I'K 


JFHQ  Organization 


SJC2E 

Functions  and 
Internal  Organization 


Plans 


Operations 


A  permanent  staff  organization 
Lead  by  a  Flag  Officer 
Mission  Tailorable 
Works  crisis  across:  Influence , 
Deter,  Coerce,  Compel  Defeat 
Allows  for  seamless 
planning  and  execution 
Applicable  across  the 
spectrum  of  conflict 


Five  supporting  teams  working  inside  an  Information  Environment 
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Common  Relevant 
Open 


GLOBA 
INFORMATION 
SOURCE 


Provide 
information 
that  is 
relevant  to 
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Changing  the  Way  We  Plan: 
Joint  Interactive  Planning 


Legacy  -  Sequential  JIP  -  Parallel  Collaborative 


Multinational  LOE  Relation  to  OC04 


Multinational 
LOE  I 


Olympic 
Impact  02 


Multinational 
LOE  II 


Olympic 
Vision  03 


Multinational 
LOE  III 


Challenge 


Nov  2001 


2002 


LOEs  are  key  to  Multinational  Participation  in 

Olympic  Challenge  2004 
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Schedule  of  Experiments 


Intelligent 
Agents  LOE 


Does  not  include  leveraged  events  like 
Global  WG,  FBE’s,  Global  Engagement, 
JCIET,  Army  TranZsformation  WG,  etc. 


Experiments  Recommended  by 
UV01  to  be  Run  in  CY2001  to 
Prepare  for  MC02 


Understanding  Quality 


Time  Period 

Weighted  %  Correct 

n 

Week  1 

93% 

21 

Week  2 

86% 

42 

Week  3 

69% 

28 

Total 

83% 

91 

-  Weight  n  =  #  Understandings  scored 

•  Correct  =  1 

•  Not  Incorrect  =  .8 

•  Incorrect  =  0 

•  Decline  of  scores  over  time  normal  as  OPTEMPO  increases 

•  High  scores  during  week  1  indicates  employment  of  core  JTF  as  part  of 
CINC  staff  effective  in  that  evidence  of  “learning  curve”  was  not  present 

•  Pre-scripted  Red  during  week  3  reduced  #  of  cases  that  could  be  scored 
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“Surrogate”  Baseline 


Quality  of  Situational  Awareness 


Surrogate  Baselines 

Day  1 

2 

3 

4 

5 

Overall 

Joint  Command 
Exercise  in  the  1980s 

.52 

.62 

.52 

.60 

.65 

.60 

Median  % 
Correct 

US  Army  Divisions 
circa  1990 1  2 

.74 

.82 

.84 

.82 

.82 

.81 

Average  % 
Correct 

Week  1 

Week  2 

Week  3 

Overall 

.93 

.86 

.69 

.83 

UV01 


Weighted 
%  Correct 


Interpretation: 

•  UV01  appears  to  have  achieved  higher  understanding  quality  than  surrogate  baselines 

•  Unlike  the  Surrogates,  understanding  quality  in  UV01  declines  over  time 

•  Time  pressure  is  greater  in  later  stages  of  UV01 

•  Goal  for  future  experimentation  should  be  stable  high  %  correct 

•  UV01  understandings  cover  full  range  of  PMESI;  Surrogates  focused  on  military  issues 


1  -  “HEAT  and  the  Warrior  Preparation  Center”,  Defense  Systems  Inc.,  August  1985 

2  -  “Use  of  ACCES  and  JESS  data  to  Support  Analyses  of  the  Relationship  of  Command  Staff  Performance  and  Battle  Outcomes”,  Defense 
Systems  Inc.,  July  1988. 
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Understandings:  PMESI 


PMESI  Count  -  Percentage  of  Understandings  for  each  category  /  %  Correct 


Time 

Period 

P 

M 

E 

S 

I 

n 

Week  1 

41%/. 83 

49%  /  .98 

8%/ 1.0 

2%  /  - 

0%  /  - 

49 

Week  2 

32%  /  .88 

53%  /  .84 

7%  /  .80 

3%  /  - 

5%  /  - 

81 

Week  3 

21%/. 33 

71%/. 74 

4%  /  - 

4%  /  - 

0%  /  - 

48 

Total 

32%  /  .79 

55%  /  .84 

7%  /  .90 

3%  /  - 

2%  /  - 

178 

n  =  total  number  of  Understandings  recorded 
=  none  scored 

•  Large  increase  in  percentage  of  military  during  execution  (Week  3)  phase 

•  Few  understandings  expressed  regarding  economic,  social  and 
infrastructure 

•  Increasing  military  percentage  over  time  trades  off  with  percentage  correct 
over  time 

•  Focus  on  military  understandings  suggests  the  presence  of  “tunnel  vision” 
which  would  threaten  PMESI  analyses  and  Effects  Based  Operations  j 


Understanding  Times 

Percentage  of  Understandings  that  considered  past, 

present  and  future  situations 


Time 

Period 

Past 

Present 

Future 

Week  1 

4% 

54% 

42% 

Week  2 

3% 

48% 

49% 

Week  3 

5% 

53% 

42% 

•  Less  than  half  of  Understandings  considered  the  future 

•  Not  ideal  to  support  planning  or  Effects  Based  Operations 

Time  Horizon  -  amount  of  time  considered  in  future  Understandings 

-  Example:  “After  today,  Red  capability  to  deploy  TBMs  will  be  reduced”  (24  hours) 

-  Mean  future  time  horizon:  2.3  Days  (game  time) 

-  Median  future  time  horizon:  2  Days  (game  time) 


Conclusion:  These  time  horizons  are  more  appropriate  for  the  tactical  than 
the  operational  level 
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Decision  Cycle  Times 


Average  Cycle  Time  (hours) 

Median  Cycle  Time  (hours) 

n 

Week  1 

29 

21.5/24.5 

6 

Week  2 

27.5 

23.5 

3 

Week  3 

21 

23 

7 

Overall 

25.1 

23/23.5 

16 

•  Cycle  times  above  represent  real  time 

•  Cycle  times  based  on  game  time  have  little  meaning  due  to  time  jumps 
between  game  days 

•  Doesn’t  account  for  events  occurring  during  jumps 

•  Doesn’t  factor  in  overnight  dead  time 


Conclusion:  The  Decision  Cycle  Times  were  stable  across  the 
experiment  rather  than  shorter  during  the  execution  phase 
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Examples  of  Long  Cycle  Times 


•  Decision 

-  Required  TMD  defense  of  Green.  First  indication  at 
261545Feb,  second  when  Green  demands  Patriots 
(171309Mar),  TMD  COA  developed  01 1 120Apr. 
Total  34  days  (Game),  68  1/2  hours  (Real) 

•  Understanding 

-  At  0 1 0900 APR  (14  May-Real)  C  JTF  states  need  to 
understand  relationship  between  Red  government  and 
JTF-S.  At  030900May  (21  May-Real)  relationship 
still  not  clear.  Total  32  days  (Game),  7  days  (Real) 

•  Cycle  times  possibly  could  have  been  shortened  had  State 
been  represented  during  initial  planning  sessions  when 
these  items  were  discussed 
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Collaboration  Summary  of  Events 

(1  of  2) 


Method 

Week  1 

Week  2 

Week  3 

Face  to  Face 

23  (50%)* 

24  (57.1%)* 

18  (32.7%) 

On-Line 

23  (50%)* 

18(42.8%) 

37  (67.2%) 

Tools** 

Whiteboard 

3  (10.3%) 

5  (25.0%) 

2  (4.8%) 

File  Sharing 

13  (44.8%)* 

13  (65.0%)* 

3  (7.3%) 

Chat 

1  (3.0%) 

0 

8  (19.5%) 

IVOX 

Not  Avail. 

Not  Avail. 

12  (19.7%) 

E-Mail 

0 

0 

5  (12.2%) 

VTC 

1  (3.0%) 

2  (10.0%) 

1  (2.4%) 

Other 

1 1  (37.9%) 

0 

10  (24.4%) 

Not  Recorded 

22 

24 

20 

-  Frequency  of  face  to  face  was  consistent  over  first  2  weeks  then 
declined  in  third  week  as  on-line  interactions  increased. 

*  Indicates  most  frequently  occurring  activities  &  tools  used. 

**  Reported  collaboration  sessions  only 


OliCAMZAIION 
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Collaboration  Summary  of  Events 

(2  of  2) 


Collaboration  Purpose 

Decision  Making 

Situation  Awareness 

Planning 

Other 

Problems  Reported 

Procedure 

Training 

System 

Personal 

Other 


Week  1 

Week  2 

1  (2.6%) 

0 

5  (13.2%) 

9(31.0%) 

28  (73.7%)* 

19  (65.5%): 

4  (10.5%) 

1  (3.4%) 

0 

0 

1  (16.7%) 

0 

4  (66.7%) 

4  (80.01%) 

0 

1  (20.0%) 

1  (16.7%) 

0 

Week  3 

Total 

3  (6.1%) 

4 

16(32.6%)* 

30 

22  (44.9%)* 

69 

8  (16.3%) 

13 

3  (23.1%) 

3 

0 

1 

10  (76.9%) 

18 

0 

1 

0 

1 

-  Planning  Collaboration  was  most  frequent,  with  SA  2nd  most  common 

-  Planning  Collaboration  Activities  decreased  over  time,  as  Situation 
Awareness  Collaboration  Activities  increased  over  time. 

-  Observers  did  not  focus  on  or  report  many  problems;  most  were  system 
problems. 

*  indicates  most  frequently  occurring  activities. 
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Collaboration  Helped 


Week  1  -  CO  A  #2  chosen  by  CINC  within  23  minutes  of  CO  A 

alternatives  being  developed.  Short  cycle  time  possible  because 
CINC  represented  in  CO  A  development  process. 

Week  3  -  CINC  disapproval  of  use  of  C-l  17’s  in  assault  operations.  This 

became  known  to  planners  during  ETO  1C  development,  enabling 
work-around  using  “chat”  feature  to  be  constructed  during  planning 
process  -  CINC  was  represented  during  planning  collaboration 
process. 

Week  3  -  CINC  representative  notices  that  CDCM  sites  not  incorporated  in 
ETO  IB  during  planning 

Week  3  -  Dynamic  re-tasking  of  ISR  assets  to  identify  hostile  intent  of 
swarming  boats  accomplished  in  15  minutes  as  a  result  of 
collaboration  between  SOF  OPS,  CINC  JIC,  JTG  OPS  Director, 
JSOTF 
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Summary  (1  of  2) 


•  Hypothesis  supported:  Quality  of 
understandings  better  than  surrogate  baseline 

•  Time  Horizon  is  short  of  ideal 

•  Absence  of  learning  curve  indication  that  core 
JTF  concept  on  right  track 

•Learning  curve  not  sustained 
•Need  to  overcome  “tunnel  vision” 

•  Long  cycle  times  for  some  events  indicates 
need  for  greater  interagency  (e.g.,  State)  play  in 
future  experiments 
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Summary  (2  of  2) 

Collaboration 

•  Helpful  in  reducing  decision  cycle  times  and  in 
avoiding  planning  errors  due  to  poor  SA 

•  Easiest  to  plan;  learning  evident  for  shared  SA 

•  Experience  with  tools  should  improve  collaboration 


OlttiAMZAllOK 
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Lessons  Learned  on  Collection 


•  Early  involvement  in  experiment  planning  is 
necessary 

•  Focused  data  collection  training  essential 

•  Need  to  observe  every  collaboration  session 

•  Use  all  of  HEAT,  not  just  part 

•  Need  dedicated  work  stations  for  HEAT 
observers 

•  Need  a  baseline  -  UV01  has  established  this 

•  Importance  of  robust  pre-test 
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Relevance  to  COBP  for  C2 

Assessment 


•  Major  parts  of  the  Code  Of  Best  Practice  are  being 
adapted  to  support  Joint  Experimentation  at  the  US 
Joint  Forces  Command.  A  publication  titled:  “A 
Code  of  Best  Practice  for  Joint  Experimentation  ”  is 
forthcoming. 
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